17. Demo 3: Optimizing k-Nearest Neighbors (k-NN) Algorithms

Understanding KNN for Stock Price Movements

The K-Nearest Neighbors (KNN) algorithm is a distance-based machine learning model used for both regression and classification tasks. This guide demonstrates using KNN to predict daily price movements of Apple stock. Here's a simplified process outline:

1. Objective

• Regression Task: Predict future one-day returns of the stock.
• Classification Task: Determine if the stock's one-day returns will move up or down.

2. Feature Engineering

• Utilize historical data to create features such as lagged returns and technical indicators like RSI (Relative Strength Index).
• Features may have null values which are resolved by dropping NAs.

3. Data Preparation

• Data Sourcing: Download Apple stock data from Yahoo Finance.
• Index Adjustment: Make the date a regular column and adjust indexes for familiarity.
• Scaling: Use a standard scaler after splitting data into training and test sets.

4. Model Implementation

• Define K (number of neighbors) as 5.
• Import KNN regressor and classifier from libraries for model training.

5. Model Evaluation

• Use cross-validation to obtain metrics like R² for regression and accuracy for classification.
• The accuracy rate provides a baseline with the training set.

This exercise serves as an introductory demonstration of how the KNN algorithm operates in both regression and classification contexts.

Analyzing and Assessing Learning Curves in Models

Key Concepts Covered:

• Demonstration of plotting learning curves using libraries like Psych learn and Matplotlib.
• Utilization of cross-validation with five folds to plot learning curves.
• Examination of training and validation score dynamics over increasing data.

Observations:

• Training scores tend to decrease as more data is introduced, while validation scores may slightly improve.
• No significant convergence of scores, indicating model underfitting.

Anomalies in Metrics:

• Negative ( R^2 ) scores signal the regression model's poor performance compared to a naive baseline.
• Classification models are preferred for tasks predicting trend directions due to the unpredictability of exact value predictions.

K-Nearest Neighbors (KNN) Adjustments:

• Decreasing neighbors to ( K=2 ) leads to overfitting; higher accuracy on the training set but poor validation scores.
• Increasing to ( K=10 ) exacerbates underfitting, with low training and validation scores.

Strategies include careful selection of model parameters to balance fitting and generalization.